DISTRIBUTION  STATEMENT  A.  Approved  for  public  release;  distribution  is  unlimited. 


Reflective  Light  Modulation  by  Cephalopods  and  Fishes 
in  Shallow  Nearshore  Habitats 

Roger  T.  Hanlon 
Marine  Biological  Laboratory 
Woods  Hole,  MA  02543  " 

phone:  508-289-7710  fax:508-289-7900  emailrhanlon@mbl.edu 

Charles  F.  Chubb 

Department  of  Cognitive  Sciences  &  Institute  for  Mathematical  Behavioral  Sciences 

University  of  California  at  Irvine 
Irvine,  C A  926 1 7 

phone:  949-824-3517  email:  cfchubb@uci.edu 

Award  Number:  N00014-06-1-0202 
http://www.mbl.edu/mrc/hanlon/coloration.html 


LONG-TERM  GOAL 

The  central  question  is:  what  optical  principles  are  used  by  opaque  organisms  to  achieve  crypsis  in 
shallow,  nearshore  marine  habitats? 

OBJECTIVES 

Camouflage  mechanisms  are  not  well  known  despite  the  general  misconception  that  they  are. 

Moreover,  quantification  of  camouflage  (especially  of  opaque  organisms)  is  particularly  wanting.  We 
have  four  objectives:  (1)  Acquire  imagery  of  camouflaged  animals  and  their  backgrounds  in  coral  reef 
and  temperate  rock  reef  environments;  (2)  collect  corresponding  in  situ  irradiance  and  reflectance  data 
from  the  animal  and  background;  (3)  Develop  a  suite  of  image  analysis  methods  to  quantify  the  type 
and  degree  of  crypsis;  (4)  Construct  a  comparative  digital  photographic  and  video  library  of  shallow- 
water  marine  animals  in  the  camouflage  categories  of  Uniform,  Mottle  and  Disruptive.  The  central 
focus  is  on  cephalopods  (octopus,  cuttlefish  and  squid)  because  they  have  the  most  diverse  and 
changeable  camouflage  patterns  known  in  biology.  Several  fishes  with  changeable  coloration  (groupers 
and  flounders  in  particular)  are  studied  comparatively. 

APPROACH 

High-resolution  digital  still  images  are  acquired  under  natural  marine  conditions.  No  flash  is  used  to 
avoid  making  artificial  shadows  from  the  flash’s  light.  A  computer-controlled  spectrometer  (adapted 
for  underwater  use)  takes  downwelling  and  sidewelling  irradiance  data  at  the  exact  time  of 
photography;  then  the  animal  reflectance  data  are  recorded  with  the  spectrometer  (in  both  gross  and 
fine  detail  on  the  animal’s  body)  so  that  color-  and  contrast-matching  can  be  quantified  in  the  digital 
images.  HDTV  video  (Panasonic  HVX200)  is  used  to  follow  foraging  cephalopods  and  fish  to 
document  (a)  speed  of  body  patterning  changes  and  (b)  the  range  of  microhabitats  that  they  encounter 
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and  the  pattern  they  choose  to  camouflage  themselves  in  each  microhabitat.  Laboratory  experiments  on 
cephalopods  are  conducted  occasionally  to  support  or  test  field  findings.  Image  analysis  programs  are 
being  developed  in  a  MATLAB  toolbox  to  enable  quantitative  analysis  of  crypsis. 

R.  Hanlon  conducts  the  field  work,  participates  in  image  analysis  and  publication,  and  directs  the  grant 
project.  Dr.  Charlie  Chubb  is  Co-Investigator  and  leads  the  efforts  on  image  analysis.  Dr.  Chubb  has 
extensive  experience  in  this  field  and  brings  a  wealth  of  quantitative  skills  to  this  set  of  tasks.  He  is 
adept  in  writing  code  for  MATLAB  and  in  applying  psychophysics  methods  to  our  laboratory 
experiments.  He  is  assisted  in  these  tasks  by  Dr.  Liese  Siemann  (postdoc  who  works  part-time  on  this 
grant)  and,  very  recently,  by  Ms.  Derya  Akkaynak  who  is  a  fourth  year  PhD  candidate  in  the 
MIT/WHOI  joint  program  in  Mechanical  Engineering  and  Oceanography.  Ms.  Akkaynak  is  co¬ 
mentored  by  Dr.  Ruth  Rosenholtz  of  MIT,  R.  Hanlon  of  MBL,  and  C.  Chubb  of  UC  Irvine.  Ms.  Justine 
Allen,  now  starting  her  third  year  in  the  Brown  University/MBL  Joint  PhD  Program  (Neurobiology), 
will  continue  to  participate  in  the  field  work  on  this  grant. 

WORK  COMPLETED 

Field  work  continued  apace  during  late  2010  and  again  in  April  2011.  RTH  completed  3  field  trips 
(total  of  35  SCUBA  dives),  acquired  ca.  1,200  high-resolution  digital  still  images  of  cephalopods  and 
fishes,  and  recorded  12  hours  of  High  Definition  TV  (HDTV)  field  video  of  crypsis  and  pattern 
change.  Briefly,  the  breakdown  is  as  follows:  (1)  Puerto  Rico,  December  2010,  to  film  Octopus 
vulgaris  using  camouflage  in  various  habitats,  including  seagrass,  soft  corals,  patch  reefs,  and  fully 
developed  coral  reefs  with  soft  and  hard  corals;  (2)  Monterey,  California,  Hopkins  Marine  Station,  to 
film  Octopus  rubescens,  a  species  with  exceptionally  complex  skin  and  patterning,  in  well  developed 
kelp  forests;  (3)  western  Turkey  (near  Izmir)  studying  the  cuttlefish  Sepia  officinalis  in  various  sea 
grass  and  sandy  environments.  New  irradiance  and  reflectance  data  (with  spectrometers)  were 
collected  in  Turkey  for  a  variety  of  applications  (see  below). 

Image  analysis  during  this  year  focused  on  developing  methods  for  predicting  which  cryptic  body 
pattern  will  be  effective  in  different  visual  habitats  and  quantifying  the  effectiveness  of  different  body 
patterns.  We  previously  developed  a  MATLAB  toolbox  that  incorporates  many  texture  and  feature 
detectors.  To  allow  flexibility,  the  program  can  be  easily  altered  to  incorporate  different  animal 
templates  or  analyses  as  needed.  The  newest  addition  to  this  program  focuses  on  comparing  the  pattern 
of  a  target  animal  to  its  adjacent  substrate.  It  automatically  extracts  a  circular  region  of  the  image 
surrounding  an  animal,  with  a  radius  that  is  proportional  to  the  size  of  the  animal  and  extracts  a  vector 
of  statistics  from  the  animal  and  the  surrounding  region.  These  statistics  can  be  incorporated  into  a 
program  designed  to  predict  body  patterns  deployed  by  cuttlefish  in  different  habitats. 

We  have  begun  devising  a  novel  approach  for  assessing  the  effectiveness  of  body  pattern.  Although 
high-fidelity  match  between  animal  and  background  may  seem  to  be  the  ultimate  camouflage,  it  is  rare 
for  two  reasons:  there  are  too  many  backgrounds  in  nature;  and  a  pattern  need  only  resemble  the 
background  in  those  statistics  to  which  the  predator  is  spontaneously  sensitive.  We  are  deriving  a 
measure  that  strives  to  reflect  the  degree  to  which  a  given  camouflage  pattern  might  be  expected  to 
elude  detection  by  any  visual  system,  including  artificial  systems  augmented  with  many  more 
dimensions  of  sensitivity  than  human  vision. 

We  are  also  developing  a  method  that  allows  accurate  representation  of  color  for  underwater  use 
because  accurate  color  representation  is  critical  for  our  image  analysis  efforts.  There  are  two 
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fundamental  limitations  to  COTS  digital  cameras:  (a)  they  are  tuned  exclusively  to  human  vision  with 
only  3  channels  represented  -  Red,  Green,  and  Blue  (with  green  emphasized  over  red  or  blue)  (b) 
manufacturers  will  not  provide  the  detailed  spectral  sensitivity  of  their  sensors.  Every  digital  camera  - 
even  two  cameras  of  the  same  make  and  model  -  will  have  a  unique  sensor  array  that  records  color 
information  differently.  Therefore  for  scientists  publishing  results  derived  from  images  on  color- 
sensitive  topics,  it  is  of  utmost  importance  to  convert  their  “unique-to-camera”  colors  to  a  common, 
standardized  color  space  so  their  work  can  be  reproduced  and  compared  with  that  taken  by  different 
cameras.  Thus,  we  are  extending  the  “terrestrial”  work  of  Stevens  et  al.  (2007)  for  use  underwater. 

RESULTS 

Crypsis  Patterning.  We  continued  to  make  advances  this  year  on  nearly  all  of  the  major  features  and 
concepts  of  animal  camouflage  (see  IMPACTS  section  below).  We  published  four  journal  papers  plus 
a  book  chapter  and  conference  paper;  most  importantly  perhaps  a  PNAS  paper  on  HyperSpectral 
Imaging  (HSI).  Second,  we  continued  to  explore  the  link  between  laboratory  experiments  on  visual 
control  of  camouflage  patterning  and  field  work  of  camouflaged  cephalopods  and  have  begun  the 
ultimate  challenge  of  predicting  what  camouflage  pattern  is  appropriate  on  different  natural 
backgrounds. 

Cryptic  patterning  in  large  and  small  fish.  Three  efforts  are  in  progress.  First,  our  fieldwork  to 
characterize  camouflage  in  large  fishes  began  with  Nassau  grouper,  Epinephelus  striatus  (graduate 
student  Anya  Watson,  University  of  Connecticut).  With  postdoc  Liese  Siemann  we  have  refined  the 
quantitative  measurement  of  pattern  types,  and  we  now  have  a  journal  paper  nearly  completed  for  these 
groupers;  we  expect  to  submit  it  within  a  few  months.  Second,  for  the  flatfish  peacock  flounder, 

Bothus  lunatus,  we  conducted  a  field  study  to  determine  the  pattern  repertoire  that  this  species  exhibits 
in  a  coral  reef  habitat;  the  results  are  excellent  and  are  nearly  written  up  for  submission  to  a  journal. 
Third,  to  demonstrate  capabilities  of  very  small  fish,  we  continue  our  study  of  the  remarkable 
camouflage  changing  capability  of  the  very  small  (1-2  inch)  slender  filefish  Monocanthus  tuckeri.  Our 
images  are  yet  to  be  fully  analyzed,  but  we  still  plan  to  produce  a  publication  in  the  upcoming  year. 

In  situ  reflectance  and  irradiance  data  with  Ocean  Optics  spectrometers.  The  basic  question  is:  when 
cuttlefish  are  camouflaged,  do  different  parts  of  their  bodies  match  the  color  and  brightness  of  the 
adjacent  background?  Two  major  strides  were  made  this  past  year.  First,  we  finished  analyzing  our 
data  from  Australia  and  have  a  paper  ready  to  submit  to  a  journal.  Second,  we  successfully  acquired  an 
extensive  data  set  on  the  cuttlefish  Sepia  officinalis  in  Turkey  in  April  2011.  These  data  will  be 
analyzed  in  the  coming  year. 

Hyperspectral  Imaging  of  cuttlefish  camouflage.  After  imaging  cuttlefish  in  the  summer  of  2010  we 
spent  this  current  year  analyzing  data.  This  paper  was  published  in  Proceedings  of  the  National 
Academy  of  Sciences  in  February  (Chiao  et  al.,  2011)  and  was  featured  as  a  Research  Highlight  in 
Nature  magazine  as  well.  The  basic  question  was:  what  does  the  camouflaged  prey  look  like  in  the 
color  space  of  the  predator’s  visual  system?  This  paper  provides  the  first  Proof  of  Principle  that 
HyperSpectral  Imaging  can  transform  studies  of  visual  perception.  Camouflaged  cuttlefish  showed 
good  color  and  pattern  match  when  viewed  by  modeled  dichromatic  and  trichromatic  fish  predators. 

Prediction  of  body  pattern  types  from  backgrounds  can  be  pursued  several  ways.  One  approach 
depends  on  the  idea  that  cuttlefish  body  patterns  can  be  quantified  and  differentiated  based  on  the  scale 
and  size  of  the  pattern  elements.  We  have  analyzed  a  series  of  laboratory  and  field  images.  A  vector  of 
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statistics  was  extracted  from  the  cuttlefish  and  from  a  circular  disk  of  substrate  surrounding  the  animal 
(the  annulus).  These  statistics  included  the  animal  and  annulus  granularity  band  energies  and  ratios 
between  these  band  energies,  Canny  and  Laplacian  of  Gaussian  edges  and  Harris  comers  in  the 
annulus,  the  presence  of  very  light/white  and  very  dark/black  regions  in  the  annulus,  and  contrast 
between  pixel  pairs  in  the  annulus  (GLCM  contrast).  When  the  contrast  in  the  annulus  remains  low 
across  a  range  of  distances  up  to  and  beyond  the  width  of  the  white  square,  cuttlefish  reliably  deploy  a 
Uniform  pattern.  For  higher  contrast  backgrounds,  we  have  discovered  that  the  ratio  of  band  2+3  to 
band  4+5  energies  (23/45  ratio)  in  the  annulus  is  highly  correlated  to  the  same  ratio  in  the  animal  in  a 
set  of  90  laboratory  and  field  images  (r  =  0.84)  (Figure  1).  This  ratio,  combined  with  other  band  energy 
ratios,  can  be  used  to  categorize  cuttlefish  patterns  as  Mottle  or  Disruptive.  Our  preliminary  prediction 
model,  based  on  granularity  band  ratios  and  GLCM  contrast,  correctly  predicts  the  body  pattern  a 
cuttlefish  will  choose  on  natural  laboratory  substrates  over  80%  of  the  time.  In  a  smaller  sample  of 
field  images,  the  same  model  correctly  predicts  the  pattern  a  cuttlefish  will  choose  close  to  70%  of  the 
time.  We  have  used  linear  regression,  coupled  with  residual  analysis,  to  identify  images  with  cuttlefish 
that  are  more  or  less  disruptive  than  predicted,  and  these  images  will  be  analyzed  further  to  investigate 
features  in  the  substrate  that  might  cause  the  animal's  pattern  to  deviate  from  what  is  predicted  by  the 
space-averaged  granularity  spectrum  and  overall  contrast  of  the  annulus. 


BAND  2  BAND  3 


Figure  1.  Linear  regression  analysis  of  the  band  23/45  ratios  of  cuttlefish  and  their  surrounding 
backgrounds.  Blue  diamonds  are  from  laboratory  images  with  different  substrates;  red  squares  are 
from  field  images;  and  orange  crosses,  purple  stars,  and  black  pluses  are  from  sets  of  lab  images 
with  different  animals  on  the  same  substrates.  The  images  on  the  right  show  an  example  of  a 
cuttlefish  from  afield  image,  along  with  the  corresponding  band  filters  used  for  granularity 
analysis,  indicating  the  sizes  of  pattern  components  for  each  granularity  band. 

The  effectiveness  of  body  pattern  has  been  assessed  using  a  “goodness-of-general-resemblance  metric” 
based  on  the  signal  detection  theory.  In  brief,  for  any  possible  value  v  that  might  be  assigned  to  a  pixel 
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of  a  texture  J,  the  histogram  k/(v)  gives  the  expectation  of  the  proportion  of  pixels  in  a  patch  of  /that 
will  take  the  value  v.  If  two  textures  /  and  J  are  identically  distributed  (i.e.,  the  joint  distributions 
characterizing  their  pixel  ensembles  are  identical),  then  it  is  obvious  that  the  following  condition  must 
hold:  for  any  image  transfonnation  T,  the  output  images  T(I)  and  T(J)  have  identical  histograms.  What 
may  not  be  so  obvious  is  that  the  reverse  implication  holds  as  well.  That  is,  if  T(I)  and  T(J)  have 
identical  histograms  for  any  image  transformation  T,  then  /  and  J  are  identically  distributed.  The  metric 
we  are  deriving  is  founded  on  this  observation.  Let  Q  be  the  patch  of  texture  filling  a  target  situated 
against  a  background  texture  R.  Suppose  the  histograms  of  Q  and  R  differ,  and  consider  the  real  valued 
function  f.  It  can  be  shown  that  the  function  maximizes  the  signal-to-noise  ratio.  By  applying  the 
function/ pixel-by-pixel  to  the  image  J,  we  derive  what  we  might  call  an  optimal  histogram-based 
“spotlight”  for  making  the  target  Q  stand  out  against  the  background  R ,  whereby  “spotlight”  we  mean 
a  real-valued  output  image  whose  mean  value  inside  Q  differs  as  strongly  as  is  mathematically  possible 
from  the  mean  value  across  the  background  R  relative  to  the  standard  deviation  of  differences  between 
the  two  regions. 


Equation  /(v)  =  Hq(v)  -  hR(v)  /  Hq(v)  +  hR(v) 


An  example  of  the  application  of  our  goodness-of-general-resemblance  metric  is  given  below  using  the 
transfonnation  functions  fTgray scale  derived  only  using  intensity  histograms  of  the  target  cuttlefish 
(0  and  its  background  (R)  pixels,  fTr/g/b  derived  using  histograms  of  red,  green  and  blue  channels 
separately,  or  JT rgb  derived  using  histograms  of  all  channels  combined.  It  is  apparent  that  when  using 
transfonnation  functions  fTgrayscale  and  fTr/g/b,  the  target  detection  is  relatively  poor  (Figures  2a, 

2b),  but  a  dramatic  improvement  can  been  seen  when  the  transfonnation  function  fT rgb  was  applied 
(Figure  2c).  This  example  highlights  the  importance  of  how  much  infonnation  colors  carry  in  an 
image.  We  expect  that  this  result  will  improve  further  when  HSI  images  are  used,  since  an  RGB 
approach  is  a  limitation  of  the  cameras  we  commonly  use,  which  is  optimized  for  the  human  visual 
system. 
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transformation 


Figure  2.  Examples  of  histogram-based  transformation  functions  for  revealing  a  target  cuttlefish. 

(a)  Transformation  function  applied  to  the  grayscale  image  does  not  strongly  reveal  the  target 
cuttlefish,  (b)  Transformation  function  applied  to  the  Red,  Green,  or  Blue  channels  also  does  not 
strongly  reveal  the  target,  (c)  Transformation  function  applied  to  the  three  color  channels  combined 

clearly  reveals  the  target  cuttlefish  as  a  bright  region. 

Habitat  specific  color  chart.  Professional  photographers  commonly  use  a  calibrated  color  chart  (e.g. 
Gretag  Macbeth  chart)  to  derive  a  transformation  from  the  camera-raw  RGB  space  (a  camera  color 
space)  to  the  XYZ  color  space  (a  device-independent  space  for  human  vision).  A  Macbeth  chart 
contains  24  color  patches  selected  to  represent  realistic  colors  that  are  likely  to  be  found  in  many  daily 
scenes  (i.e.  dark  skin,  light  skin,  foliage,  etc.)  and  each  patch  retains  a  constant  reflectance  value  when 
viewed  under  varying  terrestrial  illumination  conditions.  An  equivalent  chart  for  underwater 
photography  does  not  exist.  Furthermore,  the  fast  and  uneven  attenuation  of  long-wavelength  light 
underwater  will  not  yield  reliable  values  for  color  patches  viewed  underwater.  Thus,  for  underwater 
applications,  the  solution  is  in  situ  reflectance  spectrometry  and  derivation  of  habitat-specific  color 
charts.  We  developed  the  methodology  and  a  prototype  chart  (Figure  3)  in  western  Turkey  in  May 
2011  (Akkaynak  et  al.  2011).  We  took  various  waterproof  color  charts  underwater,  to  different  depths 
at  different  times  of  day,  and  their  reflectance  values  were  recorded  with  our  Ocean  Optics 
spectrometer  (SubSpec)  as  well  as  our  Canon  SLR  to  build  an  “underwater  color  appearance” 
database.  This  information  was  used  to  convert  camera  raw  RGB  values  to  the  standardized  XYZ 
space,  representing  the  actual  color  of  that  underwater  habitat.  Such  a  habitat-specific  color  chart 
optimizes  the  camera  raw  RGB  to  XYZ  transformation  for  that  dive  site.  A  huge  advantage  of  this 
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method  is  that  our  previously  taken  photos  (thousands)  can  be  retroactively  color-corrected  once  the 
required  “color  appearance  data”  are  collected  for  that  specific  dive  site.  When  combined  with  new 
HSI  imagery,  we  will  have  a  spectrally  rich  as  well  as  spatially  rich  image  library.  We  expect  that  other 
research  groups  may  adopt  this  methodology  since  COTS  digital  cameras  are  so  widespread. 


Figure  3.  A  habitat-specific  color  chart,  (a)  Array  of  waterproof  color  charts  photographed  at 
different  depths  at  different  times  of  day.  (b)  Prototype  habitat-specific  color  chart 

developed  for  a  dive  site  in  Turkey. 


IMPACT/APPLICATIONS 

Concept  development  of  camouflage  visual  mechanisms  continued  to  mature  this  year.  Following  the 
book  we  published  a  year  ago  entitled  Perspectives  on  Camouflage:  Art  and  Science  (Flanlon,  2010), 
we  published  a  book  chapter  summarizing  some  key  concepts  learned  from  rapid  adaptive  camouflage 
in  cephalopods  (Flanlon,  et  ah,  2011).  These  developments  have  spurred  interest  by  several  DoD 
agencies  and  discussions  are  continuing. 

Our  2011  published  paper  on  Hyperspectral  Imaging  (HSI)  was  well  received  by  the  scientific 
communities  that  focus  on  visual  perception,  coloration  and  patterning.  Fortunately  we  have  just  been 
funded  to  build  a  small  fast  HSI  unit  adapted  for  camouflage  and  signaling  studies  underwater.  This 
development,  combined  with  our  description  above  for  a  new  method  to  derive  habitat-specific  color 
charts,  will  substantially  expand  our  capabilities  in  quantitatively  assessing  camouflage  under  natural 
conditions,  and  will  provide  a  large  advantage  over  point-source  spectrometry  readings  that  have  been 
used  by  us  and  others  in  recent  years.  This  is  a  case  in  which  a  technology  breakthrough  in  a  different 
field  opens  up  new  and  more  sophisticated  avenues  of  inquiry. 
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Our  quantitative  methods  for  analyzing  animal  body  patterns  have  advanced  well  this  year  and  we 
have  taken  steps  towards  the  ultimate  goal  of  predicting  what  body  pattern  will  provide  the  best  crypsis 
in  any  given  habitat.  In  addition,  we  have  produced  a  histogram-based  transfonnation  function  for 
revealing  a  target  cuttlefish.  The  potential  applications  of  such  developments  will  be  obvious  to  most 
readers. 

TRANSITIONS 

A  few  of  our  findings  on  mechanisms  of  camouflage  are  being  considered  by  ARL,  SOCOM  and 
DARPA. 

RELATED  PROJECTS 

The  PI  has  a  4-yr  AFOSR  grant  that  focuses  on  skin  papillae  and  white  leucophores  in  cephalopod 
camouflage.  We  also  have  a  1-yr  contract  from  ARL  to  perform  lab  experiments  on  visual  background 
stimuli  that  evoke  camouflage  in  cuttlefish.  Furthermore,  we  are  one  of  the  teams  funded  on  a  4-yr 
ONR  Basic  Research  Challenge  (Rice  University  is  lead  organization);  this  is  a  materials-related  grant, 
but  we  will  address  the  biological  question  of  color-blind  camouflage  in  cephalopods  by  studying 
distributed  light  sensors  in  the  skin  (Mathger  et  al.,  2010).  Importantly,  we  were  funded  by  NSF  in 
summer  2011  to  build  an  underwater  HyperSpectral  Imager. 

C.  Chubb  is  PI  on  a  grant  from  NSF  (Perception,  Action  &  Cognition)  to  analyze  the  dimensions  of 
human  visual  sensitivity  to  various  classes  of  textures.  This  research  promises  to  have  important 
implications  for  understanding  how  camouflage  patterns  operate  to  elude  detection  by  human 
observers. 
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